Camera system for filming golf game and the method for the same

ABSTRACT

A camera system for filming a golf game comprises a camera module for capturing images; a pan-tilt head pivotally connected to the camera module and configured to rotate the camera module horizontally and vertically; a circuit board connected to the camera module and the pan-tilt head and configured to control the camera module and the pan-tilt head; a sensor unit sensing the direction, vertical angle, and horizontal angle of the camera module; and a central processing unit connected to the camera module, the pan-tilt head and the sensor unit. The present invention also provides a method for filming a golf game by using the camera system.

TECHNICAL FIELD

The present invention relates to field of golf game, particularlyrelates to a camera system and a method for filming a golf game.

BACKGROUND ART

Videos taken during a golf game are good for entertainment as well as atool to improve golfer's skill. Generally, a video of a golf game istaken by another person (generally not the golfer to play the golf game)using a camera or a smart device with a built-in camera to trace theball. Golf balls generally fly at a high speed, and it is difficult byhuman visual detection to trace a ball from a hit until the ball lands.Therefore, it is desired to provide a camera which can automaticallyfilm golfer's swing and flying of a ball all the way until the balllands. In existing technology, to determine the direction where the ballgoes after hit, will be done manually. Firstly, user needs to turn thecamera facing where he expects the ball to land after his hit and camerawill memorize the direction. Afterward, camera will consequently turn tolocate golfer. Once ball being hit and detected, the camera will turnfrom golfer to the recorded direction and film the video when balllands.

Technical Problem

An aspect of the present invention is to provide a camera system forfilming a golf game automatically. By utilizing imaging processingtechnique, camera system can capture the golfer and his/her relatedswing, detecting the ball and direction of hit including velocity of theball, and film the golf ball landing without tracing the ball.

Solution to Problem Technical Solution

In training lesson of golfer, coach will always teach student to use twolegs standing in parallel to the target line of hit. That is the standof two legs always in parallel to the target line of hit. Camera systemutilize this golf basic step to find the direction of the ball afterhitting.

In one embodiment of the invention, the camera module comprises a singlecamera. The single camera is a zoom camera and includes a zoom lens. Thezoom levels of the camera are controlled according to the velocity ofthe ball.

In another embodiment of the invention, the camera module includes twocameras. One is with prime lens and the other is with optical zoom lens.By placing the camera system in front of the golfer hitting location andwith prime lens camera facing the ball on the ground, the algorithm candetect the ball through some image object recognition techniques. Afterscanned the location of ball, it will wait for golfer to walk in theball near-by position so that the image of his/her legs can beidentified accordingly and automatically. In the process, it measuresthe distances between ball and legs, the algorithm can estimate theangle that the camera must turn in order to parallel with the targetline of hit. Once the ball is detected being hit by the prime lens, thecamera with zoom lens will turn towards the target line of hit to filmthe ball landing scene until the ball stop rolling. Zoom lens will zoomwith different levels according to the estimated velocity of the ball.

In another embodiment of the invention, only prime (fixed) lens withoutzoom lens are used. There can be an array of 3 lens which are incombination of prime lens and wide angle lenses with different focallengths. To cover the range of golf ball flight, it can be divided intothree 3 ranges. As the camera system can detect the velocity of ballbeing hit, hence the distance of the ball fly can be estimated. With thedistance known, the camera system will activate the corresponding camerato capture the video accordingly.

Advantageous Effects of Invention Advantageous Effects

The end result of the captured video is a complete record of a golfer'shit; from swing to ball landing. It applies to different situation ofgolf during golf play; from Tee off with a driver, wood/iron hit onfairway, short game on chipping, bunker shot and putting respectively.

BRIEF DESCRIPTION OF DRAWINGS Description of Drawings

FIG. 1A shows the camera module with one camera;

FIG. 1B shows the camera module with two cameras;

FIG. 1C shows the camera module with three cameras;

FIG. 2 shows the internal structure of the camera system;

FIG. 3 shows the DTL calculation from 90 degrees with the golfer;

FIG. 4 shows the DTL calculation from 45 degrees with the golfer;

FIG. 5 shows the ball detected through sensitivity zone;

FIG. 6 shows the launch angle in relation to golfer and velocity of theball captured.

FIG. 7 shows the example of DTL in relation with the legs of golfer;

FIG. 8 shows the DTL in relation to actual target towards the landingzone.

BEST MODE FOR CARRYING OUT THE INVENTION Best Mode

It will be appreciated that for simplicity and clarity of illustration,where appropriate, reference numerals have been repeated among thedifferent figures to indicate corresponding or analogous elements. Inaddition, numerous specific details are set forth in order to provide athorough understanding of the embodiments described herein. However, itwill be understood by those of ordinary skill in the art that theembodiments described herein can be practiced without these specificdetails. In other instances, methods, procedures, and components havenot been described in detail so as not to obscure the related relevantfeature being described. The drawings are not necessarily to scale andproportions of certain parts may be exaggerated to better illustratedetails and features. The description is not to be considered aslimiting the scope of the embodiments described herein.

Only one definition that applies throughout this disclosure will now bepresented.

The term “comprising” means “including”, but not necessarily limited to,it specifically indicates open-ended inclusion or membership in aso-described combination, group, series, and the like.

Embodiments of the present disclosure will be described with referenceto the accompanying the drawings.

The present invention provides a camera system for filming a golf game.As shown in FIGS. 1A-1C, the camera system includes a camera module 100,a pin 200 and a pole 300. The pin 200 and the pole 300 are employed forinserting the camera module 100 on the ground. The camera system furthercomprises a pan-tilt head 12, a circuit board 13, and a power module 14.The camera module 100 includes at least one camera 11. The pan-tilt head12 is pivotally connected to the camera module 100, and can rotate thecamera module 100 horizontally and vertically due to the actuation ofmotors 121 of the pan-tilt head 12. The motors 121 may be servo motors.The power module 14 provides power to the camera module 100. The powermodule 14 may be one or more batteries. The camera module 100 and thepan-tilt head 12 are electrically connected to the circuit board 13. Thecamera module 100 may further include a transparent cover 19 to coverand protect the camera module 100. The transparent cover 19 may beplastic, and may be circular or cylindrical.

Referring to FIG. 2, the camera system further includes a centralprocessing unit 15, a sensor unit 16, a GPS module 17, and a wirelesscommunication unit 18 arranged on the circuit board 13. The sensor unit16 senses direction, horizontal angle, and vertical angle of the cameramodule 100. The sensor unit 16 may be a gyroscope. The sound sensor unit19 is for golfer voice recording and voice commands. The wirelesscommunication unit 18 may communicate the camera module 100 with aportable device 400 of a golfer. The wireless communication unit 18 maybe a WIFI module or/and a Bluetooth module. The portable device 400 maybe a smart phone or a tablet PC. The central processing unit 15 controlsthe operation of the camera module 100, for example, controls thepan-tilt head 12 to pan the camera module 100 to a desired hittingdirection so as to film the golf game with a preset proportion of zoom,and controls the communication between the camera module 100 and theportable device 400.

Concept of Direct Target Line (DTL) is illustrated in detail with FIGS.7 and 8. During addressing, golfer stands in parallel to where he wantsto hit the ball. By connecting two points of the golfer, i.e., two legs,two feet, two knees or other two positions of the golfer towards thetarget, the DTL is formed. Preferably, the two points are two legs ofthe golfer. Most balls will fly towards the DTL direction for skilledgolfers. The camera module will film towards this direction after ballbeing hit, no matter the ball flies in line with DTL or not. Balllanding locations normally deviated from the DTL a bit, either left orright. But the zoom lens, with wide coverage of view, can film balllanding in a certain range derivate from DTL.

For skilled golfers, they may play the ball intentionally more towardsright or left e.g. draw and slice the ball etc. The DTL is the finallanding direction the golfer expects the ball to fly. It can bedifferent from his final two-legs stand location. The camera system onlyscan his/her first legs location to determine the angle for camera toturn to. After recording the angle, golfer can stand differently fromthe DTL to perform draw and slice actions.

The present invention also provides a method for filming a golf game, inaccordance with an exemplary embodiment of the camera system in thepresent invention.

In step 300, camera is firstly inserted on the ground facing thedirection of golf ball in hitting area. In step 301, the ball is beingscanned in the camera images and its location is determined. In step302, golfer walks into the hitting area that image is being scanned andgolfer body is determined. In Step 303, camera determines the legs ofthe golfer. According to the location of left and right legs as well asthe ball, three vertical lines are drawn (see FIGS. 3 & 4). D1 is thedistance between the right leg and ball as seen in camera view.Similarly, D2 is the distance between ball and the left leg. Based onratio of D1 and D2, the DTL (Direct Target Line) angle as seen in birdview can be determined.

For example, in FIG. 3, when the camera faces nearly perpendicular tothe golfer, i.e. nearly perpendicular to the DTL, D1 is almost equal toD2 which means the target direction of ball flight is around 80 degreesfrom the current angle of the camera. The ratio for D1 and D2 will bepositive and value is closing to 1. Similarly, in FIG. 4, if the camerais placed with DTL angle 45 degree, the target direction of ball flightwill be around 45 degrees from the current camera angle. The ratio of D1and D2 will be in negative value indicating the DTL angle will besmaller than the FIG. 3 case.

Sheet 1 shows the relationship of D1 And D2 and its ratio and DTL angle.

Sheet 1

TABLE 1 Left leg to Right leg to ball distance ball distance D1/D2 ratioDTL angle D1A D2a Ra DTLa D1B D2b Rb DTLb x x x x x x x x D1n D2n RnDTLn

In step 304, golfer hits the ball and camera images are being analyzedto determine if the ball has been successfully hit by the golfer. FIG. 5illustrates the detection of ball hit through a sensitivity zone, whichis an image ROI (region of interest) where high framerate images arebeing monitored. The velocity and launch angle of the ball can bedetermined by comparing the sequence of ball images captured in thesensitivity zone (see FIG. 6). With velocity measured, through Sheet 2,it can determine the duration of ball flight and distance of the balltravelled.

In step 305, the pan-tilt head 12 is controlled to pan the camera module100 to the hitting direction parallel to the angle of the DTL calculatedin step 303 so as to film the ball flying and landing, without actuallytracing the ball.

Sheet 2

TABLE 2 Flight Range of hitting Velocity duration (s) Distance (yards)of ball 1 10-20 V1 2 20-30 V2 3 30-50 V3 4  50-100 V4 5 100-150 V5 6150-180 V6 7 180-230 V7 8 230-280 V8 9 280-300 V9

In step 306, filming stops when a flight duration of the ballcorresponding to the hitting distance elapses. In this embodiment, thecamera module 100 stores relationships (Sheet 2) recording flightduration and velocity of the ball and hitting distances. Each range ofhitting distance corresponds to one flight duration. For example, if therange of hitting distance is from 150 yards to 180 yards, the velocityis measured to be V6, the flight duration of the ball will be 6 seconds.The relationships are obtained according to the golfer's experience.

Embodiment 1

Referring to FIG. 1A, the camera module 100 comprises a single camera11. The single camera 11 is a zoom camera and includes a zoom lens 111.The zoom levels of the camera 11 are controlled according to thevelocity of the ball after hitting as calculated from step 304. Sheet 3shows the relationship of velocity to zoom level. E.g. when ballvelocity V is calculated in the range of V2−V3, the zoom lens 111 willbe zoomed at 1.5× which is 1.5 times of the zoom levels.

Embodiment 2

Alternatively, in FIG. 1B, two cameras can be used, one is a prime lensor fish eye lens 211 while the other is a zoom lens 212. It can beconfigured to carry out the video task of golfer's swing and balllanding separately. The prime lens 211 is used to perform imageprocessing for golfer image, ball velocity, and angle for the zoom lens212 to turn, while the zoom lens 212 is used to take video of golfer'sswing and ball landing.

The zoom levels of the zoom lens 212 are also controlled according tothe velocity of the ball after hitting as calculated from step 304,which is similar to the zoom levels of the camera 11 in embodiment 1.

Embodiment 3

Only use prime (fixed) lens without zoom lens. There can be an array of3 lens (A, B C) which are in combination of prime lens and wide anglelenses with different focal lengths. To cover the range of golf ballflight, it can be divided into three (3) ranges e.g. A for 1-100 yards;B for 100-200 yards; C for over 200 yards etc. As the camera system candetect the velocity of ball being hit, hence the distance of the ballfly can be estimated. With the distance known, the camera system willactivate the corresponding camera to capture the video accordingly, e.g.if the ball detected flying distance is 160 yards, the camera B will beactivated to capture the video.

Sheet 3

TABLE 3 Velocity range Zoom level V1-V2 1 × X V2-V3 1.5 × X  V3-V4 2 × Xn n Vx-Vy 5 × X

The present invention also provides a method to determine the golf balllocation, golfer body and golfer's legs and generate a DTL. Given thatthe golfer is standing in front of the ball and with a typicalpreparation stand during addressing, it is known that the initialaddressing (preparation stand) of golfer have a strong and knownrelationship to his/her intended target direction of ball travelingafter hit. By capturing a sequence of images of this scene (at thepreparation stage of the golfer hitting the ball), it can estimate theintended target direction of the ball, i.e. DTL (Direct Target Line),with respect to the camera view.

A method to determine the golf ball location, golfer body and golfer'slegs and generate a DTL comprises following steps:

determining the location of the ball in the scene by using multi-modelimage correlation;

determining the location of the golfer by using Background Modeling;

cleaning the model of golfer;

using the golfer model to calculate the distance between the legs of thegolfers and to determine middle of two legs;

mapping the output of the relationship function to the DTL angle with apredetermined mapping function.

Therein,

1. Ball Localization Using Multi-Model Image Correlation

building of the ball models: capturing image of balls at different scaleand lighting conditions m(l,s), wherein l and s is lighting and scaleparameter respectively;

using the correlation function coff(im, m(l,s)), coff is a templatematching function, and search for the location with minimum coff withthe use of all the ball models. This implies the location with theminimum values will be considered as the highest possible location ofgolf ball.

2. Localization of the Golfer Using Background Modeling

After the location of the ball in the image is determined, based on thisparticular location in the image, the neighborhood area around thelocation of the ball will be selected, the size of the selected areawill be subject to the size of the ball in the image. This area will bemodeled based on the model pixel value of all location under stablecondition.

First model is the one with background and without the golfer M_(wg)

Second model is the one with background and with the golfer M_(g)

Both models are built based on the temporal filtering, when the image isstable or without any change, the machine records the image model.

Finally, by subtracting the M_(wg) by M_(g), then the result will be themodel of golfer.

3. Segmented Region Filtering

The result from the process of step 2 is the model of golfer, however itis required to eliminate some artifacts caused by noise and shadows. Thesmall blob/regions are eliminated by morphological dilation operation.

Then edge detection is performed on the image, then the unwanted one iseliminated based on threshold of its orientation.

4. Estimation of the Position of Golfer's Legs

The result from the process of step 3 is a set of edges that reflectedthe vertical line of the golfer's legs. The process approximates thelocation of the pant by measuring the occurrence of the vertical linealong the horizontal side of the camera view.

There should be three peaks if we plot the occurrence of edges againstthe x-direction of image space. Then using some histogram peak findingtechniques to search for the local peaks. In our implementation, we areusing hill-climbing techniques.

5. Relationship Function Rel(p, b) Between Golfer's Stand Position andBall

Relationship function is with the input, position of the legs, p, andposition of the ball, b. The function itself is based on the ratiobetween the width of the legs and distance from the ball to the middleof the legs.

6. Mapping Function d(Rel) Maps the Relationship Rel(p, b) to TurningAngle DTL

In this process, the machine will map the rel value to the turningangle. The mapping function is determined by a combination oftheoretical and empirical method.

1. A camera system for filming a golf game, comprising: a camera modulefor capturing images; a pan-tilt head pivotally connected to the cameramodule and configured to rotate the camera module horizontally andvertically; a sensor unit sensing the direction, vertical angle, andhorizontal angle of the camera module; and a central processing unitelectrically connected to the camera module, the pan-tilt head and thesensor unit; wherein the central processing unit is configured tocontrol the camera module, the pan-tilt head and the sensor unit to filmthe golf ball.
 2. The camera system of claim 1, wherein the centralprocessing unit is also configured to determine locations of a golf balland a golfer.
 3. The camera system of claim 2, wherein the centralprocessing unit is also configured to calculate an angle of a directtarget line (DTL) according to the locations of the golf ball and thegolfer.
 4. The camera system of claim 1, wherein the central processingunit is also configured to determine an angle for the camera module toturn to.
 5. The camera system of claim 1, wherein the central processingunit is also configured to calculate a golf ball velocity, hitting angleand hitting distance.
 6. The camera system of claim 1, wherein thecentral processing unit is also configured to control the pan-tilt headto pan the camera module to the hitting direction parallel to the angleof the direct target line (DTL).
 7. The camera system of claim 1,wherein the central processing unit is also configured to determine aflight range and a flight duration of the golf ball corresponding to thehitting distance.
 8. The camera system of claim 1, wherein the centralprocessing unit is also configured to determine whether or not a ballhas been successfully hit by the golfer by determining whether or notthe ball has passed through a sensitivity zone which is an area betweenthe golfer and a target landing point.
 9. The camera system of claim 3,wherein the central processing unit is configured to calculate an angleof a direct target line according to a location of the golf ball and afirst point and a second point on the golfer body.
 10. The camera systemof claim 9, wherein the first point on the golfer body is the left legof the golfer, the second point on the golfer body is the right leg ofthe golfer.
 11. The camera system of claim 1, wherein the camera moduleis a single zoom camera.
 12. The camera system of claim 11, zoom levelsof the single zoom camera are controlled according to the calculatedgolf ball velocity.
 13. The camera system of claim 1, wherein the cameramodule comprises a first camera and a second camera, the first camera isconfigured to perform image processing for golfer image; ball velocity;and angle for the second camera to turn, the second camera is configuredto take video of golfer's swing and ball landing.
 14. The camera systemof claim 13, wherein the first camera is a prime lens camera or a fisheye lens camera.
 15. The camera system of claim 13, wherein the secondcamera is a zoom lens camera.
 16. The camera system of claim 15, zoomlevels of the second zoom camera are controlled according to thecalculated golf ball velocity.
 17. The camera system of claim 1, whereinthe camera module comprises at least three prime lens cameras withdifferent focal lengths configured to cover different flight ranges ofthe golf ball.
 18. The camera system of claim 17, wherein the cameramodule comprises three prime lens cameras which are a first prime lenscamera, a second prime lens camera and a third prime lens camera whereina flight range of the golf ball covered by the first prime lens camerais 1-100 yards, a flight range of the golf ball covered by the secondprime lens camera is 100-200 yards, a flight range of the golf ballcovered by the third prime lens camera is more than 200 yards.
 19. Thecamera system of claim 18, wherein the central processing unit isconfigured to determine the flight range of the golf ball and activate acorresponding camera to capture a video of the ball landing.
 20. Thecamera system of claim 1, further comprising a wireless communicationunit arranged on the circuit board and configured to communicate thecamera with a portable device.
 21. The camera system of claim 1, furthercomprising a GPS module.
 22. The camera system of claim 1, furthercomprising a circuit board electrically connected to the camera moduleand the pan-tilt head.
 23. A method for filming a golf game by using thecamera system of claim 1, comprising the following steps: determininglocations of a golf ball and a golfer; calculating an angle of a DTLaccording to the locations of the golf ball and the golfer; determiningan angle for the camera module to turn to; calculating a golf ballvelocity, hitting angle and hitting distance; controlling the pan-tilthead to pan the camera module to the hitting direction parallel to theangle of the direct target line; determining a flight range and a flightduration of the golf ball corresponding to the hitting distance; andactivating the camera module to film the golf ball until the flightduration elapses.
 24. A method to determine a golf ball location, golferlocation and generate a direct target line (DTL) comprises followingsteps: determining the location of the ball in the scene by usingmulti-model image correlation; determining the location of the golfer byusing Background Modeling; cleaning the model of golfer; using thegolfer model to calculate the distance between two points on the golferbody and to determine middle of the two points; estimating a value basedon a relationship function with the input values; mapping the output ofthe relationship function to the DTL angle with a predetermined mappingfunction.
 25. The method of claim 24, wherein the multi-model imagecorrelation comprises the following steps: building of the ball modelscomprising the step of capturing image of balls at different scale andlighting conditions m(l,s), wherein l and s is lighting and scaleparameter respectively; and using the correlation function coff(im,m(l,s)), wherein coff is a template matching function, and search forthe location with minimum coff with the use of all the ball models. 26.The method of claim 24, wherein the Background Modeling comprises thefollowing steps: selecting a neighborhood area around the location ofthe ball after the location of the ball in the image is determined,based on this particular location in the image, wherein the size of theselected area will be subject to the size of the ball in the image;modeling the selecting area based on a model pixel value of all locationunder stable condition; building a first model and a second model basedon the temporal filtering, when the image is stable or without anychange, the machine records the image model, wherein the first model isthe one with background and without the golfer M_(wg), and the secondmodel is the one with background and with the golfer M_(g) subtractingthe M_(wg) by M_(g), and generating the model of golfer.
 27. The methodof claim 24, wherein the relationship function is based on the ratiobetween the width of the two points on the golfer body and a distancefrom the ball to the middle of the legs.